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1. Introduction 

Let X be a closed oriented surface of genus g > 2 and let tt = vri(X) be its fundamental 
group. Let Sp(4, R) be the group of linear transformations of preserving its standard 
real symplectic form. Consider the set A* := Hom(7r, Sp(4, M)) of group homomorphisms 
from vr to Sp(4, M). We also refer to the elements of X as representations of vr in 
Sp(4, R). Using a set of generators of vr, X can be embedded in Sp(4,M)^^, acquiring 
in this way a natural topological structure. The goal of this paper is to compute the 
number of connected components of X. 

Given a representation of vr in Sp(4, R), there is an integer associated to it, which 
is geometrically obtained by considering the flat Sp(4, R)-bundle corresponding to the 
representation and taking a reduction of the structure group to U(2) — the maximal 
compact subgroup of Sp(4, R). The degree d of this U(2)-bundle is an invariant of 
the representation, which we call the degree of the representation. Let X{d) C X be 
the set of representations of degree d. There is a Milnor-Wood type inequality ([5, 18]) 
which says that X{d) is empty unless \d\ < 2g — 2. Our main result in this paper is the 
following. 

Theorem 1.1. Let d be an integer such that \d\ < 2g — 2. Then 

(1) X{d) is non-empty and connected if \d\ <2g — 2; 

(2) X{d) has 3.2^^ + 2g — A non-empty connected components if \d\ = 2g — 2. 

The proof of (2) and the case (i = for the space of reductive representations is due 
to Gothen [8] . Recall that a representation p from tt to a real algebraic group G is called 
reductive if the real Zariski closure of p(7r) in G is a reductive group. When the closure 
coincides with G, the representation is said to be irreducible. Let d X he the 
subspace of reductive representations of vr in Sp(4,R). In Section 2 we show that the 
inclusion X^ C X induces a bijection of connected components (Theorem 2.3). In fact, 
we show, this result is valid replacing Sp(4, R) by any reductive real algebraic group G 
and TT by any finitely generated group. 

Let X^{d) be the space of reductive representations from vr to Sp(4, R) of degree d. 
This space is invariant under the action of Sp(4, R) by conjugation. The moduli space 
of representations of degree d is defined as TZ{d) := X^{d)/ Sp(4, R). The reductivity 
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condition is precisely what is needed in order for TZ{d) to be a Hausdorff space. Gothen's 
results can now be stated as follows. 

Theorem 1.2 (Gothcn [8]). The moduli space TZ{0) is non-empty and connected, and 
TZ{±{2g — 2)) has 3.2^^ -\-2g — 4 non-empty connected components. 

In this paper we settle the situation for the remaining cases. More precisely, we prove 
the following. 

Theorem 1.3. Let d be any integer satisfying < \d\ < 2g — 2. The moduli space TZ{d) 
is connected and the subspace of irreducible representations is non-empty. 

To prove this theorem, in Section 3, we follow the methods of Gothen ([8]) and Hitchin 
([11, 12]) by choosing a complex structure on X and exploting the relation between 7l{d) 
and the moduli space of polystable Sp(4, M)-Higgs bundles Ai{d). These are Higgs 
bundles (i?, $), where E = V (B V* , for V a rank 2 holomorphic vector bundle of degree 
d, and Higgs field ^ : E ^ E* ^ K of the form 



with (3 e H^iS^V ® K) and 7 G H^S^V* ® K). The basic fact is that M{d) is a com- 
plex analytic variety which is homeomorphic to TZ{d) (by results of Hitchin, Donaldson, 
Simpson and Corlette). This homeomorphism induces a homeomorphism between the 
subspace of irreducible representation in TZ{d) and the subspace of stable Higgs bundles 



By solving Hitchin equations for a Hermitian metric on V and considering the square 
of the L^-norm of $, one has a proper function on }A[d). The number of connected 
components of the local minima of this proper function gives an upper bound on the 
number of connected components of M.{d) and hence of TZ{d). These local minima have 
been characterised by Gothen [8] for any d. In particular when \d\ < 2g — 2, Gothen 
shows that the minima coincide with the subvariety J\f{d) C M.{d) for which either /3 = 
or 7 = (Proposition 3.6). Which one of the sections actually vanishes is determined 
by the sign of d. We prove that if < \d\ < 2g — 2, M{d) is connected. To show this, in 
Section 4 we study a more general situation that is of independent interest. Namely, we 
introduce a (poly)stability criterium for pairs (V,/5), where V is a rank 2 holomorphic 
vector bundle of degree d, and (3 e H^{S'^V (g) K) which depends on a real parameter 
a, thus defining moduli spaces J\fa{d). It turns out that the subvariety of minima J\f{d) 
can be identified with Moi^d) (Proposition 4.1). Our main result concerning ^/'^((i) is the 
following Theorem proved in Section 5. 

Theorem 1.4. Let —2{g ~ 1) < d < be any integer and let a > be a real number. 
The moduli space J\fa{d) is connected. Moreover, the subvariety consisting of stable pairs 
is non-empty. 




(1.1) 



in M{d). 



Prom this we deduce the following. 
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Theorem 1.5. Let d he any integer such that Q <\d\ <2g — 2. The moduli space M.{d) 
of poly stable Sp{4,M.)-Higgs bundles of degree d is connected. Moreover, the subvariety 
consisting of stable Higgs bundles is non-empty. 

This proves Theorem 1.3, which combined with Gothen's Theorem 1.2 and Theorem 
2.3 proves Theorem 1.1, since the connectedness of Sp(4, R) imphes that the number of 
connected components of X^{d) coincides with that of TZ{d). 

2. Representations of the fundamental group 

2.1. Reductive representations of 7ri(X). The results of this section and the next 
one apply also when tt is replaced by any finitely generated group. 

Let X be a closed oriented surface of genus g > 2 and let tt = tti{X) denote its 
fundamental group. Let G C GL(A^, R) be a non-compact real reductive algebraic group 
(when G is compact everything which follows in this section and the next one are also 
true for obvious reasons). Let X := Hom(7r, G) be the set of representations of tt in G, 

and let 

X'^ :— {p e X \ p{tt) C G is reductive } 

the set of reductive representations (p(vr) denotes the real Zariski closure of p(7r) in G). 
Take generators 71, . . . , 7^ of tt and consider the inclusion j : X K := End(R^)'^ which 
sends p to (p(7i), • • • , p{lk)) ^ G'^ C End(R^)'^. Consider on X the topology induced by 
j and the standard topology on the vector space E. This is independent of the choice 
of generators of vr. We take on X^ the topology induced by the inclusion X^ C X . 
Furthermore, j{X) C E is a real algebraic (affine) subvariety of G^ C End(R^)'^, whose 
equations are defined by requiring the coordinates of a point (pi, . . . , p^) G G^ to satisfy 
any relation satisfied by the generators {7^}. It follows by a theorem of Whitney [19] 
that j{X) (and hence X) has a finite number of connected components. 

Consider the adjoint action of G on Af: ii g E G and p E X then g ■ p is the rep- 
resentation defined by g ■ p{^) := gp{'j)g~^ for any 7 e tt. We consider similarly the 
diagonal adjoint action of G on E = (EndR^)'^, in such a way that the inclusion j is 
G-equi variant. One has the following. 

Theorem 2.1 (Richardson, Theorem 11.4 in [14]). A representation p E X is reductive 
if and only if the orbit G ■ j{p) C E closed in the usual topology o/E. 

Remark 2.2. If a real algebraic group acts linearly on a vector space then an orbit which 

is closed in the usual topology may fail to be closed in the real Zariski topology (in general 
its Zariski closure will consist of a finite number of orbits which are closed in the usual 
topology). This is in contrast to the situation over the complex numbers, where an orbit 
of a linear action is closed in the usual topology if and only if it is closed in the Zariski 
topology. 

It follows from Theorem 2.1 that X'^/G with the quotient topology is a Hausdorff 
space. The space TZ :— X^ / G is called the moduH space of representations of tt in 
G. 
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Our main concern is the study of the connectedness of X, and TZ. Of course, when 
G is connected the number of connected components of TZ coincides with that of . 

2.2. From reductive representations to ELrbitrEiry representations. 

Theorem 2.3. The inclusion i : X'^ C X induces a bijection of connected components. 

Proof. To prove this we need the following. (All topological notions refer to the usual 
topology in E.) 

Theorem 2.4 (Luna, Theorem 2.7 in [13]). Let G be a real algebraic group acting linearly 
on a real vector space E. Let E+ := {x e E | G • x C E Z5 closed }. For any x e E 

there is a unique closed orbit p{x) contained in the closure of G ■ x. The space of orbits 
E+ZG endowed with the quotient topology is Hausdorff, and the map p : E ^ E+/G is 
continuous. 

In [13] this theorem is stated in a slightly different form. Luna defines a G-invariant 
subset ^4 C E to be G-saturated if it contains any orbit whose closure intersects the 
closure of any orbit inside A. Then he proves that for any G-invariant closed subset 
F C E the smallest G-saturated subset of E which contains F is also closed (this is 
Property (C) of [13]). To deduce from this the continuity of p note that if G C E+/G is 
closed then p~^{C) is the smallest G-saturated set containing the closure of 5~^(G) C E, 
where g : E+ — > E+/G is the quotient map. 

Now we prove Theorem 2.3. Denote by Go C G the connected component of the 
identity. We first check that the map : ^^{X^) t^o{X) is onto. For that, take any 
p & X . By Theorem 2.4 the closure of Gq ■ j (p) contains a closed orbit Gq ■ j (po)- On the 
other hand, since Gq is connected, the closure of Gq ■ j{p) is locally arc-connected, and j 
is injective, it follows that there is a continuous map c : [0, 1] — > A' such that c(0) = po 
and c(l) = p. Finally, Theorem 2.1 imphes that po £ 

To prove that i^ is injective consider the following diagram, which commutes by the 
definition of p (see Theorem 2.4) 

X+ — '-^x 




Since Gq is connected it follows that q^, : 7ro(A'+) — > ■Kq^X^ /Gq) is a bijection, and since 
= p* o i* we deduce that is an injection. □ 

2.3. Representations of vri(X) in Sp(4, M). We use the same notation as in the pre- 
vious two sections. In particular Let E = End(M'^)''. Let now G = Sp(4, M) and let 
X :— Hom(7r, Sp(4, R)) be the set of representations of tt in Sp(4). To understand the 
geometric meaning of reductive representations of vr in Sp(4, M), consider the standard 
symplectic structure u on M''. Recall that a subspace V C is called coisotropic if 

= e I uj{v, w) - Vw e V} c V. 
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One has the following. 

Proposition 2.5. Let p : tt — > Sp(4, M) be a representation. The following are equivalent: 
(i) p is reductive; 
(a) Sp(4,R) ■ j{p) C E is closed; 

(Hi) for any coisotropic subspace V C which is preserved by p there is a splitting 

= V/V^ e (MVV © V^) (2.2) 

such that the image of p is contained in Sp(V/V-'-) x Sp(M''/V© V"*-) (one can check that 
both V/V"*- and R^/V © V"*- carry natural symplectic structures) . 

Proof. The equivalence between (i) and (ii) is given by Theorem 2.1. We just sketch the 
proof of the equivalence between (ii) and (iii) (the argument is similar to that of Lemma 

28 in [10]). Take a representation p and suppose that there is a sequence {gj} C Sp(4, R) 
which is not contained in any compact subset of Sp(4, M) and such that for any k 

{gjp{ik)gj^] ik e Sp(4, M) as j oo. 

Define Sj := g*gj., where we take the adjoint of gj with respect to the standard Euclidean 
metric in M^. Then Sj diagonialises in some basis ei, . . . , 64 with eigenvalues Ai > • ■ ■ > 
A4. Define := (e2, 63, 64) if Ai > A2 or := (63, 64) otherwise. Since Sj G Sp(4, M), V-^ 
is coisotropic. Let V be a limit of a partial sequence of {V-'}. Then, since {gjp{lk)gj^} 
is bounded, p preserves V. Now, if 7^ belongs to the adjoint orbit of 7^ then one must 
have a splitting as in (2.2). Finally, if the limit representation defined by {7^} does not 
belong to the orbit of p, then one can take a sequence {g^} in such a way that no splitting 
of the form (2.2) is preserved by p(7fc) for all k. □ 

Remark 2.6. It is clear that a reductive representation p & X is irreducible if there 
is no nontrivial splitting = Vi © V2 with both Vi and V2 symplectic subspaces ofW^ 
such that the image of p is contained in Sp(Vi) x Sp(V2). 

2.4. Chciracteristic numbers. Our first step in determining the number of connected 
components of X :— Hom(7r, Sp(4, R)) is to study the topological invariants associated 
to a representation. For any G there is a locally constant obstruction map 

o : Hom(7r, G) H\X, 7ii{G)). 

For G = Sp(4, R), we have 7ri(G) = Z; picking the standard isomorphism, the obstruction 
map associates to any p E X an integer o(p) which we call the degree of p, and which 
can be obtained more geometrically by considering the flat Sp(4, R) bundle associated 
to p. This bundle admits a reduction of the structure group to U(2) — the maximal 
compact subgroup of Sp(4, R) — , whose first Chern class is precisely the degree of the 
representation. (Note that the reduction will not be in general compatible with the fiat 
structure) . One can also show that this degree measures the obstruction to the existence 
of a Lagrangian subbundle of the flat Sp(4, R)-bundle. 

Let d be an integer; recall that X(d) C X consists of those representations whose degree 
is d. The subset X~^{d) C X{d) is given by the reductive representations of degree d 
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and TZ{d) C TZ given by X~^{d)/ Sp(4,M) is the moduli space of representations of 
degree d. Our main goal is the study of the non-emptiness and connectedness of TZ{d). 

A restriction on the possible degrees of an element in X is given by the following 
Milnor-Wood type inequality ( Turaev [18], Domic and Toledo [5]). 

Proposition 2.7. Let p be a representation of ti in Sp(4, M), and let d be the degree of 



In this section we follow the methods of Hitchin [11, 12] and Gothen [7, 8]. We refer 
to their papers for details. 

3.1. Sp(4, M)-Higgs bundles. As in the case of complex representations of tt, repre- 
sentations in Sp(4, M) are related to Higgs bundles. To recall the basic ingredients of 
this theory, we fix from now on a complex structure on X. A Higgs bundle on X is a 
pair (-E, where E \s b. holomorphic vector bundle over X and ^ : E ^ E ® K is di, 
holomorphic map, where K is the canonical line bundle of X. The Higgs bundle 
is said to be stable if for any proper subbundle F (Z E such that G F K we 

have n{F) < ij,{E), where n{F) = degF/ rkF. The Higgs bundle (E,^) is said to be 
polystable if it is the direct sum of stable Higgs bundles of the same slope IJ,{E). This 
condition appears as the requirement to solve Hitchin's equations. More precisely, we 
have the following. 

Theorem 3.1. [11, 15] Let be a Higgs bundle with degE — 0. Then E admits a 

Hermitian metric H satisfying 



if and only if (-£',$) is polystable. (Here Fh is the curvature of the unique connection 
compatible with H and the holomorphic structure of E.) Furthermore, the set of metrics 
H which solve (3.3) is convex, i.e., if H,H' are both solutions then for any real number 
t e [0, 1] the metric tH + (1 — t)H' is also a solution. 

The last statement of the theorem follows from the fact that if {E, $) is stable then 
there is a unique Hermitian metric H solving (3.3). 

The particular class of Higgs bundles that will be of relevance in relation to represen- 
tations of TT in Sp(4, M) is given by pairs {E, $), where E — V ® V*, with V a rank 2 
holomorphic vector bundle and 



p. Then 



\d\<2g-2. 



Hence X[d) is empty for \d\ > 2g — 2. 



3. Sp(4, ]R)-REPRESENTATIONS OF 71 AND HiGGS BUNDLES 



Fh + [$, $*"] = 



(3.3) 




(3.4) 



where (3 e H%S^V K) and 7 e H%S^V* ® K). 
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A Sp(4, ]R)-Higgs bundle can be regarded as a pair {V, if) where K is a rank 2 holo- 
morphic bundle and ^ = (/3,7) G H^{S^V ® K) ® H^{S^V* ® K). Two Sp(4, M)-Higgs 
bundles (y, ip) and (V, (p') are isomorphic if there is an isomorphism : V' ^ V such 
that if' = ip*(p. Let d be an integer. Let M.{d) be the moduli space of polystable Sp(4, M)- 
Higgs bundles (V, ip) such that deg V = d. By stability of {V, ip) we mean stability of the 
corresponding Higgs bundle {E, $), with E = V ® V*, and $ given by (3.4). 

The following refinement of Theorem 3.3 is necessary to relate the moduli space of 
Sp(4,M)-Higgs bundles Mid) to 7^(d). 

Theorem 3.2. A Sp{4:,'R)-Higgs bundle {V,ip) is polystable if and only if there exists a 

Hermitian metric h on V such that 

+ {(3P*- - -f*^-f) = 0. (3.5) 

where denotes the curvature of the unique connection compatible with h and the holo- 
morphic structure of V . 

Proof. Suppose that we have a metric h on V for which (3.5) holds. Then, setting 
H = h + h* we get a solution to (3.3), so that {E,^) is polystable and hence (by 
definition) {V, ip) is also polystable. 

To prove the converse, assume that {V, tp) is polystable, i.e. (i5, $) is polystable. By 
Theorem 3.3 there is a metric H on E which solves (3.3). We want to show that this 
metric can be taken of the form H = h + h* with respect to the splitting E — V ® V* . 
We can view H as a section of E <^ E , and using the splitting 

E®E*^V0V*®V0 (W)* eV* 0V* eV* ® (W)* 

we may write H — Hqq + Hqi + H^q + Hu. We claim that if H' :— Hqq — Hqi — Hiq + Hu 
is also a solution to (3.3). To prove the claim, consider a local holomorphic framing 
ei, . . . , of E whose first (resp. last) n sections give a framing of V (rcsp. V*). Let 
Mh be the 2n x 2n matrix whose {i,j) entry is (cj, e^)//- Define p := diag(i Idy, — ildy*). 
A simple computation shows that Mh' — pMnP"^- On the other hand, if we identify $ 
with a matrix by means of our framing, then we have — (M^)~^$*M^, from which 
we easily compute 

= (p-^*p)*-]p-^ - (3.6) 

(the last equality follows from = — which is a consequence of (3.4)). On the 

other hand, since we took a holomorphic framing the Chern connection takes the form, 
in our trivialisation, dn = d + {dMH)Mjj^ (see for example p. 73 in [9]), from which we 
deduce that dn' — pdnP"^ and hence Fh' — pFhP~^- This, together with (3.6), implies 
that H' is another solution to (3.3). Consequently, H" := H + H' is also a solution to 
(3.3) which satisfies H" = hy + hy* for some metric hy (resp. hy*) on V (rcsp. V*). 
To finish the argument, observe that we have an isomorphism f : E ^ E* defined as 
f{u,v) — {v,u)] now, if is a solution to (3.3), then both f*H and H* (the latter 
denotes the natural metric on E* induced by H) give solutions to the Hermite-Einstein 
equations for the pair (£;*,$*). It follows that H" + {f-^)*{H")* is a solution to (3.3) 
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which is of the form h + h* for some metric h on V. This metric gives then a solution to 



Remark 3.3. Theorem 3.2 follows also from the general Hitchin-Kobayashi correspon- 
dence proved in [3]. 

3.2. Homeomorphism between TZ{d) and Ai{d). The moduh space Ai{d) of poly- 
stablc Sp(4, ]R)-Higgs bundles can be constructed in essentially the same way as that of 
the moduli space of ordinary Higgs bundles (see §9 of [17]). Alternatively one can do 
the following: consider the map from isomorphism classes of polystable Sp(4, M)-Higgs 
bundles to isomorphism classes of polystable Higgs bundles which sends {V, if) to {E^ $); 
this map is finite to one, so there is a unique structure of reduced scheme on the set of 
isomorphism classes of polystable Sp(4, M)-Higgs bundles which is compatible both with 
the map and with the scheme structure on the set of isomorphism classes of polystable 
Higgs bundles which one gets by looking at the latter as a moduli space. 

Following either of these procedures, one ends up in particular with a structure of 
topological space on Jv[{d). We will sketch here a description of this topological structure 
on M{d) in gauge-theoretic terms. This discussion will also be relevant for us in order to 
define the proper function from which we plan to extract information about the number 
of connected components oi M.{d) (a similar study can be seen in §5 of [11]). 

Let V be a C°° complex vector bundle over X of rank 2 and degree d. Let us fix a 
Hermitian metric /i on V, and denote by the affine space of Hermitian connections 
on V. Let also Q := Q^'°(M; S'^Y -5^V*) and let ^ be the group of Hermitian auto- 
morphisms of V — the gauge group. Consider the completions of and Vl (resp. ^) 
with respect to Sobolev L\ (resp. L\) norm, and denote the resulting completions by 
the same symbols. Then the quotient ^ := (=e/ x O)/^ is a Hausdorff topological space 
(the crucial point, Hausdorffness, follows from the existence of slices of the action of ^ 
on ^ X f2). Finally, we define S{d) to be the set of gauge equivalence classes [^4, (p] e ^ 
such that 



where ip = (/9, 7). The set S{d) inherits a topology from its inclusion in By The- 
orem 3.2, the points of S{d) are naturally in bijection with M.{d), the moduli space of 
polystable Sp(4, ]R)-Higgs bundles of degree d . This bijection maps a gauge equivalence 
class [A,Lp] to the pair {V,p), where \/ is V equipped with the holomorphic structure 
defined by the (0, 1) part of the connection A. 

The following is an Sp(4, M)-version of the general correspondence between complex 
representations of tt and Higgs bundles ([11, 12, 15, 16, 6, 4]). 

Theorem 3.4. Let d be an integer. There is a homeomorphism TZ{d) = A4{d) which, 

when d ^ 0, restricts to a homeomorphism between the space of irreducible representa- 
tions in TZ{d) and the space of stable Higgs bundles in A4{d). 

Proof. In fact we prove that both Ai{d) and TZ{d) are homeomorphic to S{d). To see 
that S{d) is homeomorphic to A4{d) we can consider the latter space from the complex 



(3.5). 



□ 




(3.7) 
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analytic point of view below): consider pairs {dv,<f), where dy is a (9-operators on the 
C°° vector bundle V underlying V and E Q. Let ^ be the set of such pairs for which ip 
is holomorphic and the associated Sp(4, ]R)-Higgs bundle is polystable. We can then view 
A4{d) as the quotient of ^ by the complex gauge group. We clearly have an inclusion 
of the space of pairs {A, ip) E s^/ x Q which solve (3.7) into which descends to give a 
continuous map from S{d) to Ai{d). Theorem 3.2 now shows that this map is in fact a 
homcomorphism. 

Suppose that {E = V Q) V* , ^) represents a point in M.{d), i.e. suppose that it is a 
polystable Sp(4, R)-Higgs bundle of degree d. Prom Theorem 3.2, there is a metric h in 
V satisfying (3.5). Rewriting the equations in terms of the Higgs connection D = d^ + O, 
where A is the metric connection and 6 = ^ + $*, we see that D is a fiat Sp(4, M)- 
connection and thus defines a point in 7l{d). Notice that takes values in u(2), while 9 
takes values in the orthogonal (w.r.t. the Killing pairing) complement of u(2) C 5p(4, M). 
Conversely, by Corlette's theorem [4], every representation in 7l{d) arises in this way. 
The fact that this correspondence gives a homeomorphism follows by the same argument 
as the one given in [17] for ordinary Higgs bundles. When d 0, the solution to (3.7) 
is irreducible if and only if the corresponding Sp(4, M)-Higgs bundle is stable, hence the 
corresponding clement in TZ{d) is irreducible. When d = the solution to (3.7) in a 
polystable Sp(4, ]R)-Higgs bundle may be actually Sp(4, M)-irreducible. The reason for 
this lies in the fact that we have used the standard stability of {E, $) as the stability 
criterium for the Sp(4, ]R)-Higgs bundle. There is, however, a notion of stabihty of the 
Sp(4, ]R)-Higgs bundle in its own rigth, i.e. without using that of the corresponding Higgs 
bundle {E, $) ([3]). It turns out that these two notions are equivalent when d ^ 0, but 
when d = 0, stability of the Higgs bundle {V, ip) is only equivalent to polystability. This 
is, however, no so important for us since in Theorem 1.5, where this is used, we assume 
that dy^O. □ 

3.3. A proper function on Ai{d). We follow the ideas of [8, 12], which reduce the 
proof of the connectedness of M.{d) to proving connectedness of a smaller subspace 
N'{d) C M.{d). Let us briefly recall how this goes. 

We define, for any [A, ip] e S{d) ^ M{d), 

mA^]) Il/3||i. + ||7lli., 
where </? = (/3, 7). This expression is gauge invariant and hence descends to give a map 

/ : M{d) R. 

One can prove that / is proper, essentially by using Uhlenbeck's compactness theorem 
(see [11]). So any connected component of A4{d) contains a local minimum of / and 
hence, we have the following. 

Proposition 3.5. Let M^d) C M.{d) be the local minima of f. Then Ji4{d) is connected 
if J\f{d) is connected. 

By duahty we have M.{d) = M{—d), so it suffices to consider the case d < 0. The key 
result characterizing J\f{d) is the following. 
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Proposition 3.6 (Gothen, [8]). Suppose that d satisfies —2{g — 1) < d <0. ThenM{d) 
consists of the classes [A, ip] such that 7 = 0. 

The following sections are devoted to proving that if— 2(5r — l)<d<0, the subspace 

M{d) is connected and contains stable Sp(4, M)-Higgs bundles, thus finishing the proof of 
Theorem 1.5, and hence the main theorems stated in the introduction. To study M{d), 
we take a more general point of view and consider a moduli problem that depends on a 
real parameter a. We can then identify H{d) with the moduli space for a — Q. 

4. Quadratic pairs 

4.1. A quadratic pair on X consists by definition of a holomorphic vector 

bundle K on X of rank 2 and a holomorphic section (3 G H^{K ® S'^V)^ where K is the 
canonical bundle of X. The degree of {V, j3) is by definition the degree of V . Let o; e M. 
We say that is a-polystable if, denoting by d the degree of 

(1) d/2 < a, and if /5 = then a = d/2, 

(2) for any subbundle L cV 

(a) ii P e H^{K ^ S'^L) then degL < d — a, and in case there is equality there 
is a splitting V = L® L'] 

(b) if /5 e H^{K ® L ® V) then degL < d/2, and if there is equality then we 
have a splitting V — L' in such a way that /3 e H^{K (8) L (8) L'); 

(c) degL < a in any case, and if there is equality then there is a splitting 
V = L®L'. 

We say that (V, (3) is a-stable if equality never occurs in the inequalities required for 
polystability. 

For any rational value of a there exists an algebraic coarse moduli space Afa{d) for the 
moduli problem of families of a-polystable quadratic pairs over X of degree d. As in the 
case of the moduli space of symplectic Higgs bundles, we will not describe the algebraic 
construction of f/a{d), since for our purposes it suffices to have a topological description 
of it. To construct the topological space underlying Nai^d) one uses the same strategy 
as in Section 3.2. Namely, to use a version of the Hitchin-Kobayashi correspondence for 
a-polystable quadratic pairs (analogous to Theorem 3.2), a particular case of the more 
general correspondence proved in [3], and then use gauge theory. 

It is clear that a quadratic pair is a special case of Sp(4, ]R)-Higgs bundle. The next 
proposition says that when a = the two possible notions of (poly)stability coincide. 

Proposition 4.1. Let —2{g — 2) < d < be an integer, and let {V,(3) he a quadratic 
pair of degree d. Then, (V,/?) is a polystable (resp. stable) Sp{4, M^-Higgs bundle if and 
only if it is a 0-polystable (resp. 0-stable) quadratic pair. In other words, J\f{d) — Afoid). 

Proof. That polystability as Sp(4, M)-Higgs bundle implies 0-polyst ability as quadratic 
pair is obvious. To prove the converse, let p -.V ®V* and q -.V ®V* ^V* denote 
the projections. Assume that {V, f3) is 0-polystable as quadratic pair, and consider some 
subbundle W gV ®V* satisfying ^{W) G K ^W. We need to check that degW < 0, 
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and that equality implies splitting. We will prove the first claim, since the second one 
follows from very similar argument. Define A = p(W), B = q{W), A' — W HV and 
B' = W nV*. Using 0-polystability it is straightforward to check that 

deg{A^ + B) + deg(A + B^)<0 (4.8) 

(simply consider the subbundle A + B^ C V) . One computes 

deg(^ + B^) = deg A + deg B - deg{A^ + B) 

deg{A^ + S) = deg^ + deg B - deg(^ + B^). 

Adding up and taking (4.8) into account we get deg A + deg i? < 0. A similar argument 
implies that deg A' + deg < 0. Finally, combining these two inequalities with the exact 
sequences 

Q^B'^W^A^Q and A ^ B 

we get the required inequality deg W <Q. □ 

4.2. Familes of quadratic pairs and moduli space. If [/ is a scheme of finite type 
(resp. a complex manifold) we define a family of quadratic pairs on X parametrized 
by U to be a pair {y,B), where 1^ is a rank 2 vector bundle on U x X and B e 
H^{'Kx^ ® S'^'^) is an algebraic (resp. holomorphic) section (here t^x '■ U x X ^ X 
denotes the projection). 

The following lemma states that stability for algebraic quadratic pairs is an open 
condition in the Zariski topology, as one would naturaly expect. 

Lemma 4.2. Fix a real number a. Let U he a scheme of finite type and let (1^, B) he an 
algehraic family of quadratic pairs on X parametrized by U. For any u G U let (Xo By) 
denote the restriction of {Y , B) to X x {u}. The set 

U,{y, B) = {ueU\ {%, Bs) IS a-stahle } 

is a Zariski open subset of U . 

Proof. Fix c? G Z and let J := Jac^X. Let ^ J x X he the Poincare bundle. Define 
^ := {t^u)*{t^uxx^ ® '^*jy.x^*)i where ttu '■ U x J x X ^ U is the projection and the 
other maps tt? are the obvious analogs. Let — > J x X be a hne bundle of high enough 
degree such that the natural map 7r^7r;7*(7r}xx=^* ® '^*j^x^* ® is surjective, 

W-Ku^i-n^^x^* ® = for any j > 0, and the same thing applies for vr^xx"^ ® 
^Uxx^* ® and TT^-ft' -S?'^. Denote by I the image sheaf of the canonical map 

^®7r[/^(7r}xx<^*®^) ^ T^u*{T^hxx'^®^), and let ^ := C Trc/^Tr^^^r* ® ^) be 
the orthogonal subsheaf. Denote by Q : y* ^ K he the quadratic map induced by B 
(of course this is not a morphism of vector bundles!). The map Q induces another map 
^ : 7iu^{7iu^x'^* ®^)^ Tru*{Tr*xK ® Finally, let ^ := ^ n ^-\0). This is a 

coherent sheaf on U. Let Ud he the support of Jif with the reduced scheme structure. 
This is by definition a closed subscheme of U with the property that a closed point u e U 
belongs to Ud if and only if there is a line subbundle L C ^ with degree deg L > d and 
such that Bu € H^{L ® K). Similarly, one proves that the subscheme U'^ <Z U 
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whose closed points are the u & U such that there is a hne subbundle L C Xt with 
degree degL > d satisfying Bu G H^{L^ ® K) is closed in U, and that the subscheme 
U'J C U whose closed points are the u & U such that there is a line subbundle L G Yu 
with degree deg L > d is also closed in U. □ 

Corollary 4.3. Let U be an irreducible algebraic manifold and let {Y, B) be a (holo- 
morphic) family of quadratic pairs on X parametrized by U. The set 

Us{y, B) = {ueU\ {%, Bs) is a-stable } 

is connected. 

Proof. By Scrrc's GAGA the holomorphic family {i^, B) is induced by an algebraic family 
(r^ig, B^'s). We then have f/,(r, B) = Usif^^, ^^^s), and the latter is connected in the 
standard topology, since it is a Zariski open subset of the irreducible manifold U . □ 

The following continuous version of the algebraic notion of family will also be relevant 
for us. We say that a quadratic pair (V, j3) can be (continuously) deformed to 
another pair [V , (3') if and only if V and V are isomorphic as smooth vector bundles 
and, denoting by V the C°° complex vector bundle on X underlying both V and V and 
by dy (resp. dy') the 9-operator giving rise to the holomorphic structure of V (resp. 
V), there are continuous maps D : [0, 1] ^ f]°'^(End V) and B : [0, 1] ^ ^^(^^y ^ ^) 
such that: 

(i) D{0) = and D{1) = dy' - dy] 

(ii) 5(0) = 13 and S(l) = (3'] _ 

(iii) for any t e [0, 1] we have (By + D{t))B{t) = 0. 

Denote by (14, A) the quadratic pair defined by {dy + D{t),B{t)). We say that the 
continuous deformation (D, B) goes through a- (poly) stable pairs if for any t the 
pair (Vt,/9t) is a- (poly) stable. 

4.3. Geometry of quadratic pairs. 

4.3.1. Quadratic forms in . Let V = C^, and let x.y G V be a basis. Recall that 
the discriminant of / = ax'^ + hxy + cy'^ G S'^V is by definition Aj := (6^ — 4ac)(,T A 
yf G (A^V)^. It is clear that A : ^^V ^ (A^V)^ is a GL(V) equivariant map. Define 
the following subsets of -^^V: = {0}, &i = {f & S'^N \ f ^ 0, Aj = 0} and 
^2 = {/g52V|/^0, A;^0}. 

Lemma 4.4. (i) An element 7^ / G S'^Y belongs to &\ if and only if there exists 

^ X E Y so that f = x"^ , and in this case the span C V only depends on f . (ii) 
/ belongs to G2 if (ind only if there exist linearly independent elements x,y eY so that 
f = xy, and in this case the set of lines {Cx, Cy} only depends on f. (iii) The orbits of 
the action o/GL(V) on S'^Y are ^0, ^1, ^2- 



Proof, (i) and (ii) are easy, and (iii) follows from them. 



□ 
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One readily sees (for example, by computing the dimension of the stabilizers of GL(V) 
acting on S'^V) that &2 C S'^V is open. This imphes the following. 

Lemma 4.5. Let f G S^Y, and let pj : End(V) ^ Tf{S^Y) ^ S^Y be the map given by 
the infinitesimal action o/End(V) = LieGL(V) on S'^Y. If f & &2, then p/ is onto. 

The following lemma is straightforward. 

Lemma 4.6. Let f e S'^Y, and let pf : End(V) ^ ^X^^y) ^ ^^^^^ 
f — xy & ^2 then, in the basis {x,y) ofY, 

Lemma 4.7. If f,g G ^^V are linearly independent, then for a generic value o/ A e C 
f + \g belongs to 02- 

Proof. We have to check that does not contain any plane. This is equivalent to 
proving that the subset P^i C F{S^Y) induced by does not cointain any line. But 
P^i is given by the equation 4ac = 6^, hence is a nondegenerate conic and does not 
contain any line. □ 

4.3.2. Discriminant of quadratic pairs. Extending fibrewise the definition of the dis- 
criminant we get a quadratic map A : K <^ S'^V K'^ ® [K^Vy. For any section 
13 -.O K® S'^V we denote A o /3 by A^. Note that we always have /9"^(0) C A;^^(0). 
It follows from Lemma 4.4 that, given a pair (V, /?), we can distinguish four possibilities. 

(1) /9 = 0. 

(2) 13 7^ and A^ = 0. Then there exists a line subbundle L C so that /? e 
H\K®L^). 

(3) A^ 7^ and there exists a square root A^/^(/3) e H^{K (g) h?V)] then there exist 
two different line subbundles Li, L2 C K so that (3 G H^{K ® Li ®i L2). 

(4) There is no square root of A^; then there exists no line subbundle L C \^ so that 

/? G H^{K®L®V). 

To understand the difference between (3) and (4), observe that by Lemma 4.4 for any 
X & X such that A^(a;) 7^ one has two different lines Lx^i,Lx^2 C V^. As x moves 
around X \ A^^(O) these two fines give rise to a pair of fine subbundles of V, unless 
the monodromy around points in A^^(O) interchanges the lines. Now, the existence of a 
square root of A^ is equivalent to the triviality of the action of the monodromy on the 
pair of lines. 

Lemma 4.8. Suppose that a pair {V^(3) defined on X satisfies A = A^ 7^ and A-*^/^ 
exists. Let Li,L2 G V be the two different line bundles so that {3 G H^{K ® Li ® L2). 
Then: (i) the pair {Li,L2} is uniquely determined by V and (3; (ii) there is an exact 
sequence of sheaves 

^ Li © L2 ^ 1/ ^ Ot ^ 0; 
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here T is the divisor in X defined asT — where r{x) (resp. z{x) ) 
is the vanishing order of Is. (resp. (3) atx. 

Proof, (i) is true because the induced quadratic map 7 : y* — > X vanishes exactly at 

Lj*- U L^; (ii) follows from the next lemma. □ 

Lemma 4.9. Let (V, /3) he a pair defined on some disk ^ <Z X, and assume that A = 
7^ 0, A~-^(0) = {0}, and there is a square root A^/^ : D — C 0/ A. Let r (resp. z) be 
the vanishing order of A (resp. j3) atO. Let 6 be a coordinate in D. Then: (i) we have 
?^/2 > z; (ii) letLi,L2 G V be the line subbundles so that (3 e H'^iyK ® Li® L2); we have 
an exact sequence of sheaves: Q ^ Li ® L2 ^ V ^ O /9'^/'^~^0 0. 

Proof. Pick a trivialisation V" = © x C{x,y) so that Li = D x C{x). Then we can write 
(3 = x{ax + by), where a, b are holomorphic functions on D, and we have A = 6^. We then 
have: a — 9^ {ao+9ai) and b — 9^^'^{bo+9bi), where ao, 60 ^ and ai, bi are holomorphic. 
To prove (i), observe that 9^ divides b. (ii) follows from L2 — C{ax + by). □ 

4.4. Local study of moduli space. Take an isomorphism class [V, (3] G Na{d). By a 
result of Biswas and Ramanan [2] the Zariski tangent space oi Ma{d) at is given 

by the first hypercohomology group of the following 2-term complex: 

^y,^ : End y 9 V ^ PvA^) -^^^^ 

where pv^p is the map induced fibrewise by the infinitesimal action of End(C^) on 
along 13 C K ® S'^V. Applying hypercohomology to this exact sequence of complexes 

Q^K® s'^v[-i\ "iy^fi End y [0] ^ 

(as usual, if £^ is a sheaf, E\(f\ denotes the complex whose only nonzero term is a copy 
of E in the position —6) we get the following long exact sequence: 

^ ^(^v'./g) ^ i/°(End V) H\K ® S^V) (4.9) 
^ (^y,/?) ^ H\End V) ^H\K® S'^V) H2(^y,^) ^ 0. 

Lemma 4.10. Suppose that n^{'^v,i3) = ^^{'^v,^) = 0. Then dimMi(^y,;3) = 7{g - 1) + 
3d. 

Proof. Using (4.9) and Riemann-Roch we have 

dim (^y,^) = -x(^y,/3) = -x(Endy) + x{K (g) S'V) 

= -4(1 -g) + (3(2g -2) + 3d + 3(1 - g)) = 7(g - 1) + 3d. 

□ 

Biswas and Ramanan also prove that the points [V, (3] e Maid) at which EI° = = 
are smooth. The preceeding lemma (together with the identification Tv,j3Na{d) = 
given by Biswas and Ramanan) implies that the dimension of Ha{d) at these points is 
l{g - 1) + 3d. 
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4.5. Vanishing of If. 

Lemma 4.11. Suppose that {V, (3) is a-polystable for some a and Ap ^ 0. Then we 
have EI°(^v',/3) = unless there is a splitting V — Li® L2 in line bundles with degLi = 
degLs = d/2 and /3 e H^{K (g) Li L2). 

Proof. Thanks to (4.9) the vanishing of H° is equivalent to the injectivity of the map 
H^(EndV) — > H^{K S'^V). Suppose the map is not injective. Then there is some 
nonzero s G iJ°(End V) so that Pv,i3{s) = 0. 

Since 7^ 0, Lemma 4.6 imphes that if a; G X and s{x) ^ then s{x) has two 
different eigenvalues. On the other hand, since the characteristic polynomial of s has 
holomorphic coefficients in X, hence constant coefficients, the eigenvalues of s(x) are 
constant as x varies along X. Hence we can split V = Li (B L2 in eigensubbundles of s. 
Using again Lemma 4.6, we know that (3 G H^{K ®Li ®L2). Applying the semistability 
condition we get degLi < d/2, degL2 < d/2, and hence degLi = degL2 — d/2. □ 

4.6. Vanishing of H^. 

Lemma 4.12. Assume that ^ 0. Then B?{'^v,i3) = 0. 

Proof It suffices to show that the map H^{EndV) H^{K O S^V) in (4.9) is onto 
(this map is the one induced by p = py,/? in cohomology). By Serre duality, this is 
equivalent to the injectivity of the map H'^{S^V*) — > H^{K ® EndV^*) induced by 
tk{p*) ■ S^V* K ® End V* . But it follows from Lemma 4.5 that the map of sheaves 
p : End V ^ K ® S'^V is onto, so the map p* : K* ® S'^V* End V* is injective. Then, 
by Lemma 4.13 below, tk{p*) ■ S'^V* ^ K ® End\/* is also injective. □ 

Lemma 4.13. Let L he a line bundle on X and let A,B be coherent sheaves on X. 
Let tl '■ Hom(L* ® A,B) — > Hom(yl, L ® B) be the standard isomorphism and let f G 
Hom(L* ®A,B). Then Ker TL{f) = L ® Ker /. 

Proof. The sheaf L is flat since it is locally free, and hence the functor L ® • : Coh(X) — > 
Coh(X) is exact. But tl is the map induced by L ® • on morphisms in Coh(X). □ 

5. Proof of Theorem 1.4 

We briefly sketch our strategy. We first classify the pairs (V,/?) in three types as 
follows: 

• type A: pairs (V, (3) with A/3 = 0; 

• type B: pairs (V,/3) with A^ ^ and H^C^^V,/?) ^ 0; 

• type C: pairs [vj) with A^j 7^ and B.\'t^v,p) = 0. 

In section 5.1 we prove that any a-polystable pair of type A can be deformed to a a- 
polystable pair of type B or C, and in section 5.2 we prove that any a-polystable pair of 
type B can be deformed to a a-polystable pair of type C (in both cases, and everywhere 
below, we mean continuous deformations). Finally, in section 5.3 we prove that the 
subset of Maid) consisting of pairs of type C is indeed connected. In Subsection 5.4 
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we prove that Ma{d) contains at least one stable object, thus proving the last claim in 
Theorem 1.4. 

5.1. Pairs of type A. In this section we study the a-polystable pairs (V,/?) for which 
— 0. By Subsection 4.3.2, for any such pair there is a line subbundle L C ^ so 

that p G H^{K ® L^). Let / be the degree of L. Since ^ (3 e H^{K (g) L^), wc have 
deg K ® L"^ = 2g — 2 + 21 > 0. Also, if (V, (3) is a-polystablc for some a > then we 
must have I < d. So in this section we assume that d> — ((? — 1). 

Lemma 5.1. There exists a manifold Sd,i and a family of pairs {Y , B) onX parametrized 
by Sfi,i such that any pair with degV = d, (3 & H^{K (g) L^) and degL — I, is 

isomorphic to {'y\{s}xx, B\{s}xx) for at least one s e Sa^i- 

Proof. Let j = 2g — 2 + 21, and let fi : S^X JaCj(X) be the map which sends an 
effective divisor D to the bundle 0{D). Let also q : Pic'(X) JaCj(A) the map defined 
as q{L) — K ® L'^. Finally, let E = S^X Xjacj(x) Jac;(X). Since ji is etale it follows 
that S is a manifold. Let A : S — Jac;(X) be the projection. We define Sd,i to be the 
vector bundle over S x JaC(i_;(X) whose fibre over (c, L') is Ext^(L', A((t)) (this vector 
space has constant dimension because for any {L,L') e Jaci(A) x JsiCd-i{X) we have 
H^{X; L (g) L'*) = 0, since degL <Si L'* — 21 — d < 0). It is clear how to construct the 
family □ 

Lemma 5.2. Suppose that —{g — 1) < d < 0. There is a nonempty dense Zariski open 
subset S^i C Sd,i such that for any s E S^^ the pair (l^|{s}xx, -B|{s}xx) can be deformed 
to a pair with A 7^ through a path of a-stable pairs. 

Proof. Let us prove that generically h^{K ® L^) < h^{K ® S'^V). This means that there 

is some section (3' oi K ® S"^ which is not entirely contained in K ® LP'- Then Lemma 
4.7 implies that for generic A G C the section j3{\) = j3 + \j3' satisfies A^(;^) 7^ 0, thus 
proving the claim. 

First of all, we have x{K ® L^) < x{K ® S^V). Indeed, x{K ® S^V) = Z{g-l + d) 
and x{K <^ L'^) ^ g - 1 + 21. Now, 

g-l + 2l <3{g-l + d) <^ < {2{g - 1) + d) + {2d - 21), 

and the right hand side follows from 2{g — 1) + d > (because d > —{g — 1)) and 
2d — 21 >0 (by stability). On the other hand, since —{g — 1) < d we have 

h°{K^S^V)>x{K®S'^V)>3. (5.10) 

Now it follows from standard Brill-Noether theory [1] that for a generic L in A(S) C 
JaQ(X) we have h^{K^L'^) = max{l, xiK(»L'^)}. If h^{K®L'^) = x{K®L'^) then we are 
done, since we have x{K ® L'^) < x{K ® S'^V) < h^{K®S'^). If instead h^{K®L'^) = 1, 
then we simply use (5.10). □ 

Lemma 5.3. Suppose that — {g — 1) < d < 0. Any a-stable element in the family , B) 
can be deformed to a pair with A 7^ through a path of a-stable pairs. 
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Proof. Let S^i C Sd,i be the set of points s E S^i for which (l^s}xx, -B{s}xx) is ct- 
stable. By Corollary 4.3, S^i is connected. If S'^i is empty, then there is nothing to 
prove. Otherwise, by Lemma 5.2 the intersection S^iH S^i is nonempty, and the result 
follows. □ 

Lemma 5.4. A a-polystable element in the family {Y , B) parametrized by 
can be deformed to a pair with A through a path of a- stable pairs. 

Proof. By definition, a a-polystable element {V. j3) appearing in the family S'_(g_i)__(g_i) 
must split V = L® L', where = K and deg L' = 0. Now, S"^ = L'^ ® L (g) L' ® L'^ 
and h%K ® L'^) > xiK O L'^) = g - 1, so h%K ® L^) < h%K ® ^V). The result now 
follows from the same argument as in the preceeding lemma. □ 



5.2. Pairs of type B. By Lemma 4.11 the pairs {V, j3) which are a-polystable for some 
value of a and so that 7^ and E^ij^y^fs) 7^ are precisely those of the form 

V^Li®L2, degLi = degL2 = ci/2, (3 e H\K ^ Li ^ L2). 

(5.11) 

In this section we assume that S — d/2 is an integer. 

Lemma 5.5. Any pair (V, /3) of the form (5.11) is a-polystable for any a > 0. 

Proof. If L C Li © L2 is a line subbundle, then at least for one value of i the projection 
L ^ Li is nonzero, from which it follows that degL < deg Lj — d/2 < 0. We cannot 
have P e H^{K <Si L'^), so degL < d/2 <0 imphes a-polystability for any a > 0. □ 

Lemma 5.6. There exists an irreducible manifold Sd an a family of pairs {Y, B) on 
X parametrized by Sd such that any pair (V,/?) of the form (5.11) is isomorphic to 
('^|{s}xx,-B|{s}xx) for at least one s G Sd. 

Proof Define Sd = S'^^-'^+'^X x Jac5(X). Let tt : 5^ x X ^ Jac5(X) x Jac5(X) x X 
be the map defined by tt{D,L,x) = {^{D) ® ® L~^,L,x), where ^ : S^^^^+^X — 
^a,C2g-2+d{X) sends an effective divisor to the line bundle it represents. We have two 
Poincare bundles JSfi, ^2 — Jac5(X) x Jac5(X) x X corresponding to each Jacobian. Let 
us define Y = 7r*=5fi © t:*^2- 

On the other hand, wc have a canonical bundle =^ over S'^^~'^^'^X x X (the puUback 
of the Poincare bundle through the projection /x x Id) and a canonical section (3 G 
H^{^). Furthermore, we have an isomorphism vr^i^ ® tt*J^2 — p*^, where 

p : ^2s-2+djss: X Jac5(X) x X ^ s^^-^+dx x X is the projection. Then we define B to 
be the canonical section H^(kX^K ® 11*^1 <S> t^*^2) induced by (3. It is clear that Sd is 
irreducible and that the family {'V, B) represents every isomorphism class of pairs of the 
form (5.11). □ 

Lemma 5.7. Any a-polystable pair of the form (5.11) can be deformed through a path 
of a-stable pairs to a pair (V, /?) with A^ 7^ and EI°(^v'^^) = 0. 
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Proof. By Lcmmac 5.5 and 5.6 it suffices to prove that at least one pair of the form 
(5.11) can be deformed to a pair {y,(3) with A;3 ^ and 'EP{^v,p) = 0. We will find two 
nonisomorphic line bundles Li, L2 of degree 5 and nonzero sections j3 G H^{K Li ® L2) 
and 7 e H^{K ® Lj). Then the family {(K, A) = {Li ® L2, (3 + e'j) \ e e D} is such 
desired deformation, since (K^A) is of the form (5.11) if and only if e = 0. To find 
Li,L2, take two different points A, A' G /i(S'^^~^+'^X) and solve the following system of 
linear equations in Jac2g-2+d(-^): (i) K + Li + L2 = X, (ii) K + 2Li = A'. □ 

5.3. Pairs of type C. Let (V, P) be a pair with A — 0, and let x — (xi, . . . , Xk) 

be the vanishing locus of A. Define the type of (V, /3) (with respect to the ordering of 
the elements of x) to be 

where rj (resp. Zj) is by definition the vanishing order of A (resp. /?) at xj. Note that 
by (4.9) we always have 2zj < rj. We also have 

J^^j = deg{K^ (g) (A^^)^) = 2{2g -2 + d). (5.12) 

j 

We define the generic type to be following list of 2(2g — 2 + d) pairs 

^s- = ((l,0),...,(l,0)). 

Theorem 5.8. For any type 3^ = ((ri,zi), . . . , {rk,Zk)) there is a manifold with a 
covering (J t/j = Sg- and a family of pairs {Yi, Bi) on X parametrized by Ui for any i so 
that 

(i) if s e UiHUj then the pairs (>i|{s}xx, 5i|{s}xx) and {yj\{s}xx, Bj\{s}y,x) are iso- 
morphic; 

(ii) if ^ — ^sen ^^gj^ is irreducible; 

(iii) any isomorphism class of pairs of type ^ which is a-stable for some value of a 
is represented by at least one (V,/3) = {%\{s}xx-iBi\^s}y.x)', 

(iv) if at least one of the rj is odd, the dimension of S,y- is k + g — 1 -\- \xr\/2, where 
\xr\ denotes the cardinal of {j \ rj is odd }; this number is < 7{g — 1) + 3d, with equality 
if and only if ^ = ST^^'' ; 

(v) if all rj are even then the dimension of S^r is strictly less than 7{g — 1) + 3(i. 

Proof. Let A; be a positive integer and define = \ Amuit; where Amuit denotes the 
multidiagonal. Let us fix a type ^ = ((ri, Zi), . . . , (r^, Z}-)). 

Case 1. Suppose first that at least one rj is odd. 

Fix some x = {xi, . . . , Xk) G B,y, let Xr = {xj G x | r j is odd }, and let p : X' ^ X be 
the 2 : 1 covering ramified at xr. Take a pair {V, (3) with A^^(O) = x and with type ^. 
Since xr^0 there is no square root of A/3. Let {V, (5') = p {V, /?), and let A' = A/3/. By 
construction there is a square root of A', so there are line bundles Li,L2 C V satisfying 
p' G H^{p*K^Li^L2). 
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Let a : X' ^ X' the Galois transformation. There is a canonical lift a : V ^ V 
which leaves 13' invariant and swaps Li and L2 (this follows from (i) in Lemma 4.8 and 
the fact that if a*Li = Li then Li would descend to a line bundle Lq ^ X satisfying 
13 e H^{K <Si Lo<SiV), hence should have a square root). Denoting L = Li it then 
follows that P' e H^(p*K a*L). 

By Lemma 4.8 we have an exact sequence 

0^ L®a*L^V' ^Ot,^ 0, (5.13) 

where is the following divisor on X': 

In order for the sequence to be c-equivariant one has to take the natural action of a on 
0{Tx) obtained by identifying the sections of 0{Tx) with meromorphic functions on X'. 
So we have 

2degL = degV - ^ r,- + 2 = 2d -Y^Vj + 2j2zj. 

j j j j 

Consider the divisor 

p{y)=Xj£x\xR p{y)=Xj£xji 

in X'. Since the numbers Zj describe the vanishing order of it follows that p*K L 
a*L = 0{Dx). 

So begining from the pair {V, (3) we have constructed a line bundle L on X' of degree 
d - Ei + Ej satisfying p*K ®L®(t*L^ 0{Dx). 

Conversely, we can recover the isomorphism class of {V, l3) as follows. Let us denote 
Exti(0T., © a*L)l^, C Exti(OT., L © ct*LY 

the set of elements giving locally free extensions. The cr-equivariant extensions of the 
type (5.13) are classified by the elements of ^■x.i^{OT^,L © (t*LY, and the isomorphism 
classes of cr-equivariant vector bundles V which are obtained through extensions of the 
form (5.13) are classified by 

Ext^(0T., © a*L)l^, _ Ext^(0r., I^h.e 
Aut(L © a*LY X Aut(CTj Aut(L) x Aut(CTj ' 
and the latter quotient has a unique element, represented by the class of any extension 

L®0{T^) ^Ot^®L = Ot^^Q. (5.14) 

On the other hand, the section (3' (and hence j3) is uniquely determined (up to multi- 
plication by scalars) by the fact that it induces an isomorphism p*K®L®a*L = 0{Dx). 

Let d^r = d — Y^- rj/2 + Zj. We are now going to study the set 

<S^ {L e Jacd^(X') | p*K ®L®aL^ 0{D^)} . 

The following lemma implies that Sx is a torus of dimension — 1) -\- d. 
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Lemma 5.9. Let r : JaCrfy(X') Jac2dy(-^') be the map which sends L to L®aL. The 
image of t coincides with i&C2d^{X'Y , and the fibres of t can be identified to the torus 

<So = {A e Jaco(X') I A ® crA ^ O} , 

which has dimension g — 1 -\-\xr\/2. 

Proof. It is clear that the fibres of r are the orbits of the action of Sq on Jac^^ {X') given 
by restricting the canonical action of Jaco(X'). 

For any L G ^a,C2d^{X'Y there exists a square root L}^"^ G JaCrfy(X')'^, which conse- 
quently satisfies t(L^^'^) = L. It follows that Jac2d^{X'Y C Imr. The other inclusion is 
obvious, so we get Jac2(iy (X')'^ = Imr. 

Let us now compute the dimension of Sq. Recall that we have an identification 
Jaco(X') = H^(X';'R)/H^(X';Z), which is cr-equi variant, since it comes from identi- 
fying Jaco(X') with the set of gauge equivalence classes of fiat U(l)-connections on the 
trivial bundle on X'. Let H\X'; ]R)=^ be the eigenspace of a corresponding to the eigen- 
value ±1. We then have TcSq = H\X';RY and H\X;R) ^ H\X';R)+. So (using 
complex dimension everywhere) 

dim<So = dimi/^(X';M)- = dimi/^(X';R) - dim//^(X;R) 

^g{X')-g = g-l + \xn\/2. 

Here we have used Hurwitz' formula x(X') = 2x{X) — \xji\ to deduce g{X') — 1 — 
x{X')/2 = 1 - x{X) + \xr\/2 ^-l + 2g+ \xr\/2 □ 

Define n : S^r — > B^r to be the fibration whose fibre over x G B^r is Sx- As x moves 
along B^ the divisors xr sweep a divisor in B^ x X which we denote by xr. Let 
p : X'^ — > B,y X X be the 2 : 1 covering ramified along x^, and let a : X'^ X'.j be the 
Galois transformation. Let Da^ (resp. T^) be the divisor in X'^ swept by the divisors 
Dx (resp. Tx) as x moves along By-. 

There is a universal bundle ^ Sg- X'a; for which there is an isomorphism of 
bundles over S^r Xb^- X'gr 

0{Dy) ^ K ®^ ®a*^ (5.15) 

(we omit the puUbacks). Now take a covering oi by open sets Ui admiting trivialisa- 
tions 

=Sf (8) OT:^\uixx — Or^luixx- 

Using each of these trivialisation we can obtain families of pairs on Ui x X beginning 
from the extensions (5.14). By the preceeding arguments it follows that any isomorphism 
class of pairs {V, /?) is represented in the family {Y, B). It is also clear that the dimension 
of Sy is A; -I- — 1 -I- 1x^1/2. Since B^r and the fibres of S^r — > B^r are connected, we 
deduce that is connected. 

Finally, observe that ^s*^" falls in the case we are now considering. In this situation, 
k = \xr\ = 2{2g -2 + d), so that 

dim5^gen — 7(g — 1) -|- 3d. 
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When ^ 7^ ^gen ^j^en both k and are < 1{2g — 2 + ri), and one does not have 
equaUty in both cases (i.e., h = \xii\ — 2[2g — 2 + d) only holds for = ^^'^^). 

Case 2. Suppose that all the rj are even. Let {V, f3) be a pair of type 3^ and vanishing 
locus X & Bar. There is a square root A^/^ of A^, so we have two line bundles Li, L2 on 
X, an exact sequence 

^ Li © L2 ^ y ^ c»T, ^ 0, 

with Tr = ^j(rj/2 — -2:^)0;^ (see Lemma 4.8), and an isomorphism 

0{D^) c^K®Li®L2, (5.16) 

where is the divisor zjXj in X. It follows that the isomorphism classes of pairs 
(V, (3) with type ^ and vanishing locus x are in 1 — 1 correspondence with line bundles 
L = Li E Jac(X) (indeed, once Li has been chosen, we set L2 = 0{Dx) ® ® L^-^) 
and a choice of a class in 

Ext^(C>T.,Li©L2)free 



Aut(Li © 12)" X Aut(CTj ' 



On the other hand, Ext^ (Ot^, Li © L2)free is the following open subset of Ext^ (Ot^, Li © 
L2) 

Ext\OT^,L, © L2)free = Ext^Or^, L, © L2) \ U /* Ext\OT, L, © L2), 

where the union is over all torsion sheaves Ot admitting a surjection / : Ot^ — » Ot and 
/* : Ext^(OT, Li © L2) ^ Ext^{OT,,Li © L2) is the map induced by /. 

To compute Ext^(CT^, Li © L2) we use Serre duality to obtain Ext^(Or^, Li © L2) = 
Hom(Li © L2, Ot^ K), and deduce from it that 

dim Ext^ (Ot, , Li © L2) = 2 rj - 2zj. 

On the other hand, since we have dimAut(OT) = ~ 2^j, we obtain: 

Ext^(C)T.,Li©L2)free 



dim 



* .X * N = fVr,- -2^,') -2 < Vr. -2 = 2(2^-2 + ci) -2. 

Aut(Li © L2y X AutiOrJ \^ ' V " ^ ^ ^ ^ ^^^ -^g^ 



Now, if the pair {V, j3) is a-polystable for some value of a then we necessarily have 
degLi <d/2 and degL2 <d/2. (5.16) implies that 

deg Li = deg(i:'3,) -degK - deg L2 

= ^^, + 2-25-degL2, 

so semistability implies that 

deg Li e [m^, M,^] := Zj + 2-2g- d/2, d/2 
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Consequently we can define 

:= □ S\X), 

m,y<d<M,y 

where S"^ —>■ Jacd(X) x is the bundle whose fibre over {L,x) is 

Aut(L ® O(D^) ® X-i (g) L-i) X Aut(e)rJ ' 

By (5.18) this has dimension dim*^*^ < 5(5^ — 1) + 2d + k — 1. Finally, since all zeros of 

Ag have even order (hence > 2) we have k < 2{g — 1) + d = deg(ii'^ ® (A^\^)^)/2 so 
dim S''^ < 7{g — 1) + 3d — 1. The definitions of the local families 1^ and Bi can be given 
exactly as in Case 1. □ 



5.4. Existence of stable objects. Here we prove that for any —2{g — 1) < d < and 
a >0 there is a a-stable pair (V, P) with deg V = 0. More concretely, we show that there 
exists a a-stable pair of degree d and generic type, i.e., such that Ap has simple zeroes 
(and consequently j3 never vanishes). By the resuhs in Subsection 5.3, the isomorphism 
classes of pairs of degree d of generic type are in 1 — 1 correspondence with choices of: 

• a (2 : 1) covering p : X' ^ X , ramified along 2{2g — 2 + 0?) different points of X; 

• a line bundle L on X' of degree —2{g — 1) such that 

p*K ®L®a*L^C, (5.19) 
where a : X' ^ X' is the unique nontrivial automorphism of p. 

The pair (V, /?) corresponding to one such choice is related to this data as follows: there 
is an exact sequence 

L®a*L^p*V ^Or^Q, (5.20) 

where Or is the structure sheaf of the ramification locus, andp*/? e H^{p*K ® a*L) 
is a nonvanishing section giving rise to the isomorphism (5.19). 

Define, for every k G Z, Jac^ := {L G JaCfc(X') | a*L = L} and Jac^ := {L G 
JaCjk(X') I a*L = It is easy to check that for any pair p,q & Z the map given by 

tensor product Jac^ x Jac~ — > JaCp+q(X') is a covering map. 

Pick some integer S > —2{2g — 2 + d) and define 

Ws := G Jac_2(5-i)-5(X') I h%^) > 0}. 
This is a complex submanifold of Jac_2{g-i)-s{X') , and we have 



dim < -2{g-l)-S. 



(5.21) 
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Take a square root K^^'^ e Jac(X) of K. Let also Q be the quotient Jac_2(g_i)_5 / Jac^^, 
and consider the diagram 

c 

/ ^ 

JacQ ^ Q 

where a maps =Sf to its class [=Sf] in Q and / maps Lq e Jacp to \p*K^/'^ ®Lq]. The map 
/ is a covering, hence dimQ = dim JaCg , and this dimension is equal to 

dimTcJaCg = dimH\X'-M)- = ?,{g-l) + d. 

(see the proof of Lemma 5.9). It is then straightforward to check that for any 5 > 
—2{2g — 2 + d) we have dimPV^ < dimQ. So, if we set 

W:^ U Ws, 

S>-2(2g-2+d) 

which is a finite union because of (5.21), then c(W) ^ Q. Hence there is some Lq G JacQ 
such that /(Lo) ^ c{W). Let L := p*K^I^ Lq, and let {V,(5) be the pair constructed 
using the ideas in Subsection 5.3. 

We now prove that (V, /5) is a-stable. First of all, by construction does not admit 
a square root. Hence, (F, (3) could only be unstable if there were a line subbundle h.<zV 
such that degA > c > 0. In this case, by (5.20), we would have a diagram 

> L e (J*L ^p*V ^ Or ^ 

Oj')" " i" 

^p*A ^0, 

in which T is a torsion sheaf which injects into Or by i. Consequently, we have 

5 := degM > degp*A - \Or\ = 2c - 2(2^ - 2 + d) > -2(2^ - 2 + d). 

On the other hand, M e Jac^ and the map (j, j') is a invariant, hence j' = a*j. And, 
since (j, j') is an inclusion, it follows that j : M ^ L is a nonzero holomorphic map. 
Hence ^ := M* ®L G W^. But this implies that /(Lq) = [L] G c{Ws), contradicting our 
assumption. So there does not exist any subbundle A C V with degA > 0, thus (V,/3) 
is a-stable. 
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